The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks

Brains process information in spiking neural networks. Their intricate connections shape the diverse functions these networks perform. In comparison, the functional capabilities of models of spiking networks are still rudimentary. This shortcoming is mainly due to the lack of insight and practical algorithms to construct the necessary connectivity. Any such algorithm typically attempts to build networks by iteratively reducing the error compared to a desired output. But assigning credit to hidden units in multi-layered spiking networks has remained challenging due to the non-differentiable nonlinearity of spikes. To avoid this issue, one can employ surrogate gradients to discover the required connectivity in spiking network models. However, the choice of a surrogate is not unique, raising the question of how its implementation influences the effectiveness of the method. Here, we use numerical simulations to systematically study how essential design parameters of surrogate gradients impact learning performance on a range of classification problems. We show that surrogate gradient learning is robust to different shapes of underlying surrogate derivatives, but the choice of the derivative’s scale can substantially affect learning performance. When we combine surrogate gradients with a suitable activity regularization technique, robust information processing can be achieved in spiking networks even at the sparse activity limit. Our study provides a systematic account of the remarkable robustness of surrogate gradient learning and serves as a practical guide to model functional spiking neural networks

Talking science, online

Traditional scientific conferences and seminar events have been hugely disrupted by the COVID-19 pandemic, paving the way for virtual forms of scientific communication to take hold and be put to the test

Co-dependent excitatory and inhibitory plasticity accounts for quick, stable and long-lasting memories in biological networks

The brain’s functionality is developed and maintained through synaptic plasticity. As synapses undergo plasticity, they also affect each other. The nature of such ‘co-dependency’ is difficult to disentangle experimentally, because multiple synapses must be monitored simultaneously. To help understand the experimentally observed phenomena, we introduce a framework that formalizes synaptic co-dependency between different connection types. The resulting model explains how inhibition can gate excitatory plasticity while neighboring excitatory–excitatory interactions determine the strength of long-term potentiation. Furthermore, we show how the interplay between excitatory and inhibitory synapses can account for the quick rise and long-term stability of a variety of synaptic weight profiles, such as orientation tuning and dendritic clustering of co-active synapses. In recurrent neuronal networks, co-dependent plasticity produces rich and stable motor cortex-like dynamics with high input sensitivity. Our results suggest an essential role for the neighborly synaptic interaction during learning, connecting micro-level physiology with network-wide phenomena

Nonnormal amplification in random balanced neuronal networks

In dynamical models of cortical networks, the recurrent connectivity can amplify the input given to the network in two distinct ways. One is induced by the presence of near-critical eigenvalues in the connectivity matrix W, producing large but slow activity fluctuations along the corresponding eigenvectors (dynamical slowing). The other relies on W being nonnormal, which allows the network activity to make large but fast excursions along specific directions. Here we investigate the tradeoff between nonnormal amplification and dynamical slowing in the spontaneous activity of large random neuronal networks composed of excitatory and inhibitory neurons. We use a Schur decomposition of W to separate the two amplification mechanisms. Assuming linear stochastic dynamics, we derive an exact expression for the expected amount of purely nonnormal amplification. We find that amplification is very limited if dynamical slowing must be kept weak. We conclude that, to achieve strong transient amplification with little slowing, the connectivity must be structured. We show that unidirectional connections between neurons of the same type together with reciprocal connections between neurons of different types, allow for amplification already in the fast dynamical regime. Finally, our results also shed light on the differences between balanced networks in which inhibition exactly cancels excitation, and those where inhibition dominates.Comment: 13 pages, 7 figure

Developmental depression-to-facilitation shift controls excitation-inhibition balance

Changes in the short-term dynamics of excitatory synapses over development have been observed throughout cortex, but their purpose and consequences remain unclear. Here, we propose that developmental changes in synaptic dynamics buffer the effect of slow inhibitory long-term plasticity, allowing for continuously stable neural activity. Using computational modeling we demonstrate that early in development excitatory short-term depression quickly stabilises neural activity, even in the face of strong, unbalanced excitation. We introduce a model of the commonly observed developmental shift from depression to facilitation and show that neural activity remains stable throughout development, while inhibitory synaptic plasticity slowly balances excitation, consistent with experimental observations. Our model predicts changes in the input responses from phasic to phasic-and-tonic and more precise spike timings. We also observe a gradual emergence of short-lasting memory traces governed by short-term plasticity development. We conclude that the developmental depression-to-facilitation shift may control excitation-inhibition balance throughout development with important functional consequences

Percutaneous reduction and fixation of intraarticular calcaneal fractures

Objective: Percutaneous reduction by distraction and subsequent percutaneous screw fixation to restore calcaneal and posterior talocalcaneal facet anatomy. The aim of this technique is to improve functional outcome and to diminish the rate of secondary posttraumatic arthrosis compared to conservative treatment and, secondly, to reduce infectious complications compared to open reduction and internal fixation (ORIF). Indications: Sanders type II-IV displaced intraarticular calcaneal fractures. Contraindications: Isolated centrally depressed fragment. Contraindications: Patients who are expected to be noncompliant. Surgical Technique: Four distractors (Synthes™) are positioned, two on each side of the foot, between the tuberosity of the calcaneus and talus and between the tuberosity and cuboid. A distracting force is given over all four distractors. A blunt drifter is then introduced from the plantar side to unlock and push up any remaining depressed parts of the subtalar joint surface of the calcaneus. The reduction is fixated with two or three screws inserted percutaneously. Postoperative Management: Directly postoperatively, full active range of motion exercises of the ankle joint can start, with the foot elevated in the 1st postoperative week. Stitches are removed after 14 days. Implant removal is necessary in 50-60% of patients. Results: Between 1999 and 2004, 59 patients with 71 fractures were treated by percutaneous skeletal triangular distraction and percutaneous fixation. A total of 50 patients with 61 fractures and a minimum follow-up of 1 year were available for follow-up. According to the American Orthopaedic Foot and Ankle Society Hindfoot Score, 72% had a good to excellent result. A secondary subtalar arthrodesis was performed in five patients and planned in four (total 15%). Böhler's angle increased by about 20° postoperatively. Sagittal motion was 90% andsubtalar motion 70% compared to the healthy foot

Extended Lateral Approach for Intra-articular Calcaneal Fractures: An Inverse Relationship between Surgeon Experience and Wound Complications

The current reference standard for the treatment of displaced intra-articular calcaneal fractures is open reduction and internal fixation using an extended lateral approach. In the present retrospective study, we evaluated the results of a consecutive series of patients treated in the same fashion from June 2005 to September 2011 using a subcuticular single-layer closure technique. We also determined the risk factors for the development of wound complications and the rate of wound complications. Also, we assessed which patient, fracture, and surgical characteristics affected these complications. During the 75-month study period, we operated on 53 displaced intra-articular calcaneal fractures in 50 patients using the extended lateral approach. The incision was closed using the subcuticular technique in 49 cases (92.45%). In the subcuticular closure group 2 (4.1%) deep infections and 2 (4.1%) superficial wound complications (1 dehiscence and 1 infection) occurred. Wound edge or flap necrosis was not encountered. The use of bone-void filler and the experience of the surgical team were significantly (p < .001 and p = .026, respectively) associated with the occurrence of wound complications. The subcuticular single-layer suture technique is a suitable closure technique in the treatment of displaced intra-articular calcaneal fractures. It was associated with a low complication rate combined with the extended lateral approach. The effect of bone void fillers on the incidence of complications should receive more attention in future research. The association between wound complications and the experience level of the surgical team supports the need for centralization of this complex injury

Visualizing the similarity and pedigree of NEURON ion channel models available on ModelDB

Are you lost in a sea of ion channel models